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ABSTRACT 

This  paper  describes  a  remote  vision  system 
applicable  to  the  teleoperation  of  unmanned 
ground  vehicles.  The  features  of  the  remote 
vision  system  are  presented  along  with  a 
description  of  its  components:  a  sensor  suite, 
a  head  slaved  platform  and  a  head  mounted 
display. 

An  emphasis  is  placed  on  the  remote  platform 
dynamics.  Performance  goals  are  discussed 
and  nominal  head  trajectories  generated  for 
use  in  a  simulation  study  which  was 
conducted  to  determine  if  these  goals  are 
achievable.  Simulation  and  experimental 
results  demonstrate  that  the  electromechanical 
pan  and  tilt  prototype  approaches  the  desired 
performance  goals.  An  analytical  formulation 
of  the  pan  and  tilt  dynamics  is  also  given. 

1.  INTRODUCTION 

The  Naval  Ocean  Systems  Center  (NOSC)  has 
developed  and  utilized  remotely  operated 
systems  for  numerous  applications  involving 
subsea,  land-based  and  airborne  vehiclesli3. 
Several  of  these  systems  have  been  developed 
with  a  characteristic  referred  to  as  remote 
presence  or  telepresence.  Telepresence  systems 
allow  human  operators  to  project  their 
sensory,  motor  and  cognitive  skills  to  remote 
and  potentially  hostile  environments  while 
giving  them  the  sensation  of  being  present  at 
the  remote  location1'4,3.  The  faithful 
reproduction  of  sensory  information,  and  the 
design  of  transparent  remote  system  controls, 
determines  the  degree  of  telepresence 
experienced  by  the  operator. 


A  critical  attribute  of  a  telepresence  system  is 
its  remote  vision  system.  NOSC  has 
developed  several  land-based  remotely 
controlled  systems  which  incorporate 
telepresence  vision  concepts.  These  include 
the  Advanced  Teleoperator  Technology 
Vehicle  (ATTV)1  and  the  Teleoperated 
Vehicle  (TOV)6  (shown  in  Figure  1) 
developed  for  the  Marine  Corps’  Unmanned 
Ground  Vehicle  Program.  Both  systems 
provide  a  stereoscopic  display  to  the  operator 
using  head  mounted  cathode  ray  tube  (CRT) 
monitors,  and  allow  the  operator  to  aim 
stereoscopic  video  cameras  using  head 
motions.  This  configuration  permits  natural, 
hands-off  vision  system  control,  freeing  the 
operator’s  hands  and  attention  to  attend  to 
other  tasks. 

2.  REMOTE  VISION  SYSTEM 

Video  technology  is  widely  used  in  the 
operation  of  remotely  contolled  land  vehicles. 
Due  to  the  inherent  tradeoffs  involved  in 
selecting  the  visual  field  of  regard  and 
resolution,  most  remotely  operated  systems 
incorporate  some  means  of  changing  the 
camera  direction,  in  addition  to  using  variable 
focal  length  lenses.  Joystick  control  of  a  pan 
and  tilt  mechanism  is  one  of  the  most 
common  methods  employed  for  controlling 
camera  position.  Some  systems  utilize 
multiple  fixed  cameras  pointing  in  different 
directions  in  conjunction  with  multiple 
monitors  to  display  the  images  to  the 
operator.  Other  methods,  such  as  steering 
slaved  cameras7  where  the  pointing  direction 
of  the  cameras  is  slaved  to  the  direction  of 
the  steering  wheel,  have  been  studied  in 
attempt  to  simplify  the  operator  interface. 
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Figure  1.  UNMANNED  GROUND  VEHICLE:  TELEOPERATED  VEHICLE  (TOV) 


The  telepresence  concept  attempts  to  make  the 
display  and  control  interfaces  to  the  operator 
as  transparent  as  possible.  This  allows  the 
operator  to  sense  the  remote  environment 
much  as  if  he  were  actually  there. 
Stereoscopic  video  is  employed  to  furnish  the 
operator  a  three  dimensional  image  of  the 
remote  environment  which  enhances  his  depth 
perception  capabilities8.  Certain  off  road 
obstacles  have  been  noted  to  be  undetectable 
without  the  use  of  three  dimensional  imagery 
(e.g.,  certain  drop-offs  and/or  ledges). 

As  an  example  of  the  potential  benefits  of  a 
telepresence  display,  consider  the  situation  of 
an  operator  of  a  remote  vehicle  approaching  a 
crossroad.  With  limited  field  of  regard 
provided  by  the  camera(s)  one  can  only  see  a 
short  distance  down  the  approach  avenues 
without  changing  camera  direction.  The 
operator  using  a  telepresence  system  would 
naturally,  almost  subconsciously  from  years  of 
experience  in  operating  a  car,  turn  his  head  to 
look  to  the  left  and  right  to  ensure  clear 
passage.  Placed  in  a  similar  situation,  the 
driver  using  a  joystick  aimed  vision  system 
would  need  to  consciously  go  through 
several  distinct  steps  (stopping  or  slowing  the 
vehicle,  panning  the  camera  left  and  right  and 
then  back  to  the  driving  frame,  and  resume 


travel)  to  ensure  no  cross-traffic. 

The  telepresence  vision  system  described  here 
is  very  similar  to  prior  ones  developed  at 
NOSC  with  the  exception  that  the  remote  pan 
and  tilt  platform  uses  electromechanical 
actuation  as  opposed  to  prior  systems  which 
arc  electro-hydraulic. 

The  major  features  of  this  remote  presence 
system  are: 

*  Isomorphic  coupling  of  the  sensor  suite  to 
the  operator’s  head  movement. 

*  Providing  situational  awareness  through 
visual  and  aural  cues. 


*  Stereoscopic  vision  for  enhanced  obstacle 
detection  in  the  remote  environment. 


*  Providing  the  operator  with  the  ability  to 
project  voice  into  the  remote  locale. 


This  full  telepresence  audio  visual  system 
consists  of  four  primary  elements:  a  head 
mounted  display,  a  head  orientation  sensor,  a 
remote  sensor  suite  and  a  pan  and  tilt  servo¬ 
mechanism.  The  complete  system  is  shown 
in  Figure  2.  _ 
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3.  HEAD  MOUNTED  DISPLAY 


The  head  mounted  display  (HMD)  developed 
at  NOSC  is  utilized  for  this  application9.  The 
HMD  incorporates  two  1"  diameter,  high 
resolution  CRT’s.  The  CRT’s  convert  video 
signals  from  the  remote  environment  into  a 
visual  image  presented  to  the  operators  eyes 
via  optical  eyepieces.  The  two  2-dimensional 
images  must  be  closely  matched  in  size  and 
orientation  so  that  the  operator’s  eyes  can 
comfortably  fuse  the  two  images  into  single 
3-dimensional  (stereoscopic)  virtual  image. 
Full  stereo  overlap  with  a  40°  horizontal  field- 
of-view  is  achieved  using  this  binocular  visual 
display. 

An  Aviator’s  Night  Vision  System  mount  is 
used  for  attaching  the  image  sources  to  a 
helmet  worn  by  the  operator.  Eyepiece  and 
mount  adjustments  allow  different  drivers  to 
adjust  the  HMD  for  their  individual  head 
geometry  and  visual  acuity  (except  for 
astigmatism). 


The  helmet  also  provides  audio 
communication  features  (headsets,  microphone 
and  push  to  talk  switch)  which  are  required 
in  most  telepresence  applications.  Head 
orientation  is  obtained  using  an 
electromagnetic  sensor  mounted  on  top  of  the 
helmet.  This  sensor  provides  three- 
dimensional  noncontact  sensing  of  the  head’s 
orientation  within  orthogonal  magnetic  fields 
generated  from  an  electromagnetic  source 
mounted  directly  above  the  operator’s  head. 

4.  REMOTE  SENSORS  &  PLATFORM 

The  remote  platform  is  comprised  of  a  sensor 
suite  and  an  electromechanical  pan  and  tilt. 
This  section  reviews  the  components  used  on 
the  sensor  suite,  the  performance  requirements 
and  the  head  tracking  servomechanism. 

4.1  REMOTE  SENSOR  SUITE 

The  remote  sensor  suite  consists  of  two 
cameras  arranged  in  a  stereo  pair 
configuration,  a  pair  of  microphones  and  a 
speaker.  As  shown  in  Figure  3,  the  CCD 
cameras  are  mounted  on  optical  alignment 


Figure  2.  REMOTE  PRESENCE  AUDIO-VISUAL  SYSTEM 


stages  to  facilitate  pitch,  roll  and  convergence 
angle  adjustments  required  for  proper 
stereoscopic  video  operation. 

The  microphone  and  camera  placement 
locations  were  selected  based  upon  human 
factor  data  supplied  by  Mil-Std-1472C  for 
95th  percentile  ground  troops  and  aviators. 
The  microphones  are  positioned  in  rubber 
pinnae  which  are  mounted  on  the  platform. 
The  use  of  artificial  pinna  separated  the  same 
distance  as  a  human’s  provides  the  operator 
with  natural  binaural  hearing  to  distinguish 
the  directionality  of  aural  cues  at  the  remote 
locale.  A  speaker  is  mounted  on  the  pan  and 
tilt  base  for  remote  communications. 

The  remote  cameras  are  high  resolution 
monochrome  charge  coupled  devices  (CCD’s). 
Auto-iris  lenses  with  a  40°  horizontal  field-of- 
view  are  utilized  to  match  to  the  field-of-view 
of  the  HMD.  This  provides  the  operator  with 
a  1:1  spatial  (orthoscopic)  correspondence  of 
the  remote  scene  and  allows  him  to  remotely 
view  objects  much  as  if  he  were  actually 
present  at  the  sensor  suite  location.  That  is, 
the  apparent  visual  angular  subtense  of  an 
object  appears  to  the  operator  to  be  the  same 
as  if  his  eyes  were  located  at  the  remote 
camera  location. 


4.2  DYNAMIC  REQUIREMENTS 

Initial  head  tracking  requirements  were 
estimated  as  a  velocity  of  360°/sec  and  an 
acceleration  of  lOOCf/sec2,  with  no  overshoot. 
These  estimates  are  based  on  average  values 
of  what  was  considered  nominal  head 
azimuthal  velocity  and  acceleration. 
However,  the  actual  peak  velocities  and 
accelerations  attained  during  head  motion  are 
quite  higher.  Recently,  experiments  have 
been  conducted  which  measured  pilot  head 
motions  during  simulated  air-to-air  combat 
scenarios10.  The  pilot  wore  a  Kaiser  Agile 
Eye  HMD  and  attained  peak  velocities  and 
accelerations  of  344°/sec  and  2452°/sec2, 
respectively,  in  elevation  and  6017sec  and 
4753°/sec\  respectively,  in  azimuth.  These 
experiments  imply  that  the  pan  and  tilt 
servomechanisms  should  have  approximately 
twice  the  estimated  bandwidth  to  track  peak 
velocities. 

For  simulation  purposes,  hypothetical 
nominal  head  trajectories  have  been  generated. 
A  typical  azimuthal  trajectory  based  on  a 
ramp-step-ramp  velocity  profile  is  shown  in 
Figure  4a.  A  smooth  velocity  profile  was 
calculated  using  a  quintic  polynomial  and  the 
initial  and  final  conditions  of  head  motion  in 
azimuth.  The  trajectory  generated  by  these 
conditions  and  structure  is  shown  in  Figure 
4b. 


Figure  3.  SIDE  AND  FRONT  VIEW  OF  THE  REMOTE  SENSOR  SUITE 


Figure  4.  HYPOTHETICAL  HEAD  TRAJECTORIES: 

(a)  RAMP -STEP -RAMP  and  (b)  QUINTIC  POLYNOMIAL 


4.3  THE  REMOTE  PLATFORM 

A  prototype  electromechanical  pan  and  tilt 
has  been  developed  as  shown  in  Figure  2.  A 
DC  motor  attached  to  a  Dojen  epitrichoidal 
gearhead  actuates  each  joint  This  particular 
raotor/gearhead  combination  can  achieve  no- 
load  velocities  in  excess  of  500°/sec.  The 
gearhead  is  a  high  precision  device  which 
contains  essentially  no  backlash  and  a  small 
degree  of  hysteresis.  Position  and  velocity 


feedback  are  both  provided  for  control. 

Each  axis  is  idependently  controlled  utilizing 
proportional  gain  control  with  velocity 
damping.  This  is  implemented  using  a  servo 
amplifier  to  close  the  loop.  Integral  control 
action  is  unnecessary  since  the  human 
operator  acts  to  integrate  out  steady  state 
errors.  A  general  block  diagram  of  the  basic 
control  scheme  with  the  coupling  dynamics  is 
shown  in  Figure  5. 


Figure  5.  BLOCK  DIAGRAM  OF  PAN  &  TILT  CONTROLLER 


To  evaluate  the  effect  of  nonlinearities  on 
position  tracking  a  simulation  was  performed. 
The  simulation  incorporates  dynamic  coupling, 
amplifier  saturation  and  actuator  saturation. 

A  full  analysis  of  the  dynamics  for  a  general 
pan  and  tilt  mechanism  was  undertaken.  The 
joint  reference  coordinate  systems  for  the  pan 
and  tilt  mechanism  are  assigned  according  to 
the  Denavit-Hartenberg11  representation  as 
shown  in  Figure  6. 

The  equations  of  motion  can  be  written  for 
the  applied  joint  torques  in  the  general  state 
vector  form 

T(t)=D(©)©+h(Q,0)+c(©)+B0 


where 

T  =  n  x  1  generalized  torque  vector  applied 
at  the  n  joints 

D  =  n  x  n  symmetric  inertial  matrix 
h  =  n  x  1  nonlinear  Coriolis  and  centrifugal 
force  vector 

c  =  n  x  1  gravity  loading  force 
vector 

B  =  n  x  n  diagonal  damping 
coefficient  matrix 

These  terms  can  be  found  using  the 
Lagrange-Euler  formulation12  with  additional 
assumptions  imposed  about  the  symmetry  and 
mass  distributions  of  the  links 

D(  1 , 1  )=Ilyy+I2„Sin2©2+l2>yCos20z 

D(U)=0 

D(2,l)=0 

D(2,2)=I^ 


JOINT  REFERENCE  COORDINATE 
SYSTEMS 


h(l)=2(I2jol-I2n)Cos©2Sin02  ©^ 
h(2)=(I2)7-I2JCos02Sin©;  ©,2 
c(l)=  0 

c(2)=  -mjgY^Sin©, 

B(1,1)=B, 

B(l,2)=0 

B(2,l)=0 

B(2,2)=B2 

where 

Iijj  =  moment  of  inertia  of  link  i  about  the  jj 
axis 

m2  =  mass  of  link  2  (tilt  platform) 

Y^  distance  to  CG  of  link  2  along  Y  axis 
0,  =  angular  displacement  of  the  i*  link 
B,  =  damping  of  link  i 

The  assumptions  incorporated  in  this 
derivation  are  that  the  origins  of  the  axes  in 
the  horizontal  plane  are  located  at  the  center- 
of-gravity  of  each  link  and  that  these  axes  are 
also  coincident  with  the  plane  of  symmetry 
for  the  mass  distribution  of  each  link.  If 
each  link  is  designed  such  that  it  is  balanced 
and  symmetric  with  respect  to  its  own  (local) 
coordinate  system  then  the  mass  products  of 
inertia  become  zero  thereby  alleviating  inertial 
coupling  effects. 

For  the  tilt  link  ,  it  was  not  assumed  that  the 
joint  be  coincident  with  the  center-of-gravity 


along  the  vertical  Y2  axis.  This  allows  for 
the  overhung  load  condition  which  exists  in 
the  prototype.  Appropriate  modifications  need 
to  be  incorporated  in  all  I2iJ  inertial  terms 
should  an  overhung  load  condition  exit.  This 
is  easily  accomplished  using  the  parallel  axis 
theorem.  If  an  overhung  load  is  non  existent 
then  the  gravity  force  vector  c  need  not  be 
incorporated  into  the  analysis.  Though 
inertial  coupling  is  alleviated  with  this  design 
approach,  Coriolis  torque  loadings  remain 
imparted  to  joint  1  (pan)  and  centrifugal 
torques  are  present  at  joint  2  (tilt). 


4.4  RESULTS 

Simulations  were  run  using  azimuth  and 
elevation  trajectories  similar  to  the  step- ramp- 
step  velocity  profile  as  shown  in  4a.  Figure 
7  shows  the  simulation  results  for  position 
tracking  and  nonlinear  torque  magnitudes  for 
simultaneous  azimuth  and  elevation 
commands.  More  specifically,  Figure  7c 
shows  the  magnitudes  of  the  Coriolis, 
centrifugal  and  gravitational  torques 
normalized  with  respect  to  the  available  (stall) 
torque  at  the  gearhead  output  shaft.  As 
expected,  the  overhung  load  places  the 
greatest  demands  on  available  motor  torque. 
The  gravitational  loading  is  approximately 
12%  of  the  available  torque.  Both  the 
Coriolis  and  centrifugal  torque  loadings  are 
smaller  than  the  gravitational  torque  and  may 
be  considered  as  dynamic  disturbances  to  the 
control  loop  as  they  have  little  effect  on 
tracking  performance.  However,  these 
disturbances  are  inversely  proportional  to  the 
gear  ratio.  If  direct  drive  motors  or  low  gear 
ratios  are  employed  the  Coriolis  and/or 
centrifugal  effects  may  become  significant. 

Experiments  were  conducted  on  the  pan  and 
tilt  prototype  to  determine  its  bandwidth  and 
tracking  capabilites.  The  servo  amps  on  the 
prototype  were  current  limited  to  50%  of  their 
rated  capability  thus  limiting  the  system  to 
25%  of  its  available  power.  This  was  done 
as  a  measure  of  protecting  the  amplifiers. 
Additional  power  is  lost  to  binding  due  to 
misalignment  between  the  motor  and  gearhead 
caused  by  the  adapter. 

The  input  and  feedback  position  signals  at  the 


servo  amp  were  measured  to  characterize  the 
pan  and  tilt.  Thus,  head  tracking  delays 
induced  by  the  position  sensor  were  not 
included  in  the  experimental  data.  Test 
results  under  these  conditions  showed  that  the 
prototype  pan  and  tilt  has  a  bandwidth  of 
approximately  0.5  Hz.  in  both  axes. 
Response  to  a  step  input  demonstrated 
classical  second  order  behavior  with  less  than 
3%  overshoot. 

The  pan  and  tilt  was  commanded  by  various 
sine  wave  inputs  to  observe  dynamic  tracking 
lags.  A  0.8  Hz  sine  wave  cycling  over  132° 
resulted  in  a  tracking  lag  of  approximately 
100  msec.  Under  human  head  motion 
control,  cycling  over  180"  at  approximately 
0.3  Hz,  dynamic  lags  were  roughly  40-60 
msec.  The  expermentai  outcomes  confirmed 
the  simulation  results. 


5.  CONCLUSIONS 

A  head  coupled  sensor  platform  has  been 
designed  to  operate  in  conjunction  with  a 
head  mounted  display.  This  results  in  a 
remote  presence  audio-visual  system  which 
can  help  provide  a  natural  driving 
environment  for  the  remote  operation  of 
unmanned  ground  vehicles. 

High  resolution  stereoscopic  vision  and  the 
ability  to  track  a  human’s  head  motion  are 
key  elements  of  the  remote  presence  audio¬ 
visual  system.  The  prototype  head  coupled 
sensor  platform  was  designed  to  provide  high 
resolution  video  on  a  moderate  bandwith 
electromechanical  pan  and  tilt  prototype. 

The  prototype’s  performance  approaches  the 
initial  goals  and  agrees  with  simulation 
results.  However,  recent  experiments  have 
indicated  that  bandwidth  requirements  are 
twice  as  large  as  initially  estimated.  An 
improved  design,  currently  under 
consideration,  will  incorporate  components  to 
meet  this  requirement. 
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Figure  7.  SIMULATION  RESULTS: 
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